In information technology, many processes, applications and environments face growing demand for analyzing data records. Such environments and applications can be e.g. computer networks, network management systems, intrusion detection systems, credit card transaction systems, flight booking management systems or warranty claim processing systems.
Data analysis according to the present invention comprises in particular data mining. Data mining can be used to detect anomalies or data aggregations in the data records. Detecting anomalies or aggregations in data records may be used for protection purposes (e.g. in intrusion detection systems), for detecting fraud (e.g., in financial transactions), for control purposes or for general observations (e.g. to improve manufacturing processes by analyzing warranty claims)
For analyzing a given situation in such technical systems or applications, data records extracted from events are used. Examples for events are pieces of traffic in a network, alarms of intrusion detection systems or information about credit card transactions, phone calls, flight bookings or warranty claims.
The quality of protection, control and observation relies on the quality of information extracted from the volume of data records. The volume of data records of such applications is usually so high that it cannot be handled efficiently by human beings. In “Jiawei Han and Micheline Kamber, Data Mining, Concepts and Techniques, Academic Press, 2001” an overview of known data mining methods is provided.
An overview of known intrusion detection systems is given in “Kathleen A. Jackson, INTRUSION DETECTION SYSTEM (IDS) PRODUCT SURVEY, Version 2.1, Los Alamos National Laboratory 1999, Publication No. LA-UR-99-3883, Chapter 1.2, IDS OVERVIEW”.
Intrusion detection systems analyse activities of internal and/or external users for forbidden and/or anomalous behaviour. They are based on the assumption that misuse can be detected by monitoring and analysing network traffic, system audit records, system configuration files or other data sources.
False alarms, appearing in large numbers, are a severe problem because investigating them requires time and energy. If the load of false alarms in a system gets high, human system administrators or security personnel might be too overwhelmed to be able to identify the true alarms.
In “Klaus Julisch, Marc Dacier: “Mining Intrusion Detection Alarms for Actionable Knowledge, KDD 2002” a method for analyzing intrusion detection alarms is described. The method is based on Attribute-Oriented Induction (AOI) algorithms. AOI algorithms are a set-oriented database mining method which generalizes data records attribute-by-attribute, compresses it into a generalized relation, and extracts from it the general features of data. In other words, attribute-oriented induction summarizes the information in a database by repeatedly replacing specific attribute values with more general concepts according to user-defined concept hierarchies or generalization rules.
Due to the ever increasing amount of electronic data records, there is a need for analysis methods with improved efficiency, accuracy and/or speed.
In light of the foregoing, it would be desirable to provide an improved computer implemented method, a computer program and a computer system for analyzing data records in which various problems associated with known systems discussed above can be alleviated.